Skip to content

client/net: Use litep2p as the default network backend#8461

Merged
lexnv merged 36 commits intomasterfrom
lexnv/litep2p-default
Jun 3, 2025
Merged

client/net: Use litep2p as the default network backend#8461
lexnv merged 36 commits intomasterfrom
lexnv/litep2p-default

Conversation

@lexnv
Copy link
Copy Markdown
Contributor

@lexnv lexnv commented May 7, 2025

Litep2p Becomes the Default Network Backend

This PR finalizes the litep2p integration and makes it the default network backend for substrate-based chains.

Litep2p Improvements

After the stabilization, a forum post will follow with up to date information and more accurate measurements of the live impact of litep2p.

CPU Usage Reduction

Litep2p consumes roughly 2x less CPU than the libp2p alternative. This frees up resources for other usecases (subsystems) and enables running nodes on more cost-efficient hardware.

This metric has been collected by the networking::libp2p-node metric of a live Kusama validator. This represents the CPU time spent on polling the networking task. Litep2p CPU consumption is on the left, using roughtly 1.3x CPUs, while libp2p on the right uses roughly 2.9-3x CPUs:

Screenshot 2025-05-26 at 15 23 22

This metric has been collected by the NodeExporter of a live Kusama validator. Litep2p CPU consumption is on the left, using roughtly 230 CPU units, while libp2p on the right uses roughly 350 CPU units. This makes litep2p ~1.52 times more effiecient:

Screenshot 2025-05-26 at 15 24 33

DHT Improvements and Authority Discovery

Litep2p is able to discover peers faster via the Kademlia protocol than libp2p. This behavior manifests in faster discovery times for validators. For context, libp2p discovers 1K DHT records (authority records) in approximately 10 minutes, while litep2p discovers them in just 2.5 minutes (for more info see #7077 (comment)).

This will improve issues we've seen with libp2p that causes validators to not receive rewards:

Stable Sync Peers

Litep2p presents a more stable peer count in comparison with the libp2p backend. This ensures we can sync up faster than libp2p to the tip of the chain. In an older experiment, litep2p syncs to the tip of the chain in 526s, compared to 803s for libp2p. The stability of connections shows improvements for other protocols as well:

Screenshot 2025-05-26 at 15 01 59

The previous image shows on the left the litep2p version and on the right the libp2p version.

Revert Kusama Enablement

This PR reverts #7866. Litep2p is now enabled by default, we don't need to selectively enable it on different chains.

Litep2p 0.9.5

This release primarily focuses on strengthening the stability of the websocket transport. We've resolved an issue where higher-level buffering was causing the Noise protocol to fail when decoding messages.

We've also significantly improved connectivity between litep2p and Smoldot (the Substrate-based light client). Empty frames are now handled correctly, preventing handshake timeouts and ensuring smoother communication.

Finally, we've carried out several dependency updates to keep the library current with the latest versions of its underlying components.

Fixed:

  • substream/fix: Allow empty payloads with 0-length frame (#395)
  • websocket: Fix connection stability on decrypt messages (#393)

Changed:

  • crypto/noise: Show peerIDs that fail to decode (#392)
  • cargo: Bump yamux to 0.13.5 and tokio to 1.45.0 (#396)
  • ci: Enforce and apply clippy rules (#388)
  • build(deps): bump ring from 0.16.20 to 0.17.14 (#389)
  • Update hickory-resolver 0.24.2 -> 0.25.2 (#386)

Fix peerset reserve only mode

This has been move in PR: #8650 for ease of reviewing.
The PR rejects non-reserved peers in the reserved-only mode of the litep2p notification peerset.

lexnv added 3 commits May 7, 2025 14:52
This reverts commit bcc272a.

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv lexnv self-assigned this May 7, 2025
lexnv and others added 5 commits May 7, 2025 15:15
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
lexnv added 4 commits May 16, 2025 15:08
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv lexnv added T0-node This PR/Issue is related to the topic “node”. A4-backport-stable2409 labels May 16, 2025
lexnv added 13 commits May 16, 2025 20:08
The test is failing too shortly in the overloaded CI.

```
output: 2025-05-16 16:02:40 polkadot-omni-node
...
2025-05-16 16:02:49 [Parachain] 🏆 Imported #3 (0xf558…5881 → 0x3909…8e9c)
// final log line
```

It is clear 10 seconds are not enough for this test and will randomly
pass if the CI is not overloaded.

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@paritytech-release-backport-bot
Copy link
Copy Markdown

Created backport PR for stable2412:

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-8461-to-stable2412
git worktree add --checkout .worktree/backport-8461-to-stable2412 backport-8461-to-stable2412
cd .worktree/backport-8461-to-stable2412
git reset --hard HEAD^
git cherry-pick -x f6d18b73c4138cd834c6979be6eed291e63c6f17
git push --force-with-lease

@paritytech-release-backport-bot
Copy link
Copy Markdown

Git push to origin failed for stable2503 with exitcode 1

paritytech-release-backport-bot Bot pushed a commit that referenced this pull request Jun 4, 2025
…8650)

This PR rejects non-reserved peers in the reserved-only mode of the
litep2p notification peerset.

Previously, litep2p ignored completely the reserved-only state while
accepting inbound connections. However, it handled it properly during
the slot allocation phase.
- the main changes are in the `report_inbound_substream` function, which
now propagated a `Rejected` response to litep2p on the reserved-only
state
- in response, litep2p should never open an inbound substream after
receiving the rejected response
- the state of peers is not advanced while in `Disconnected` or
`Backoff` states
  - the opening state is moved to `Cancelled`
- for consistency purposes (and fuzzing purposes), the
`report_substream_opened` is more robustly handling the `Disconnected`
state
- while at it have replaced a panic with `debug_assert` and an instant
reject

## Testing Done
- started 2 nodes in Kusama and Polkadot with litep2p
- added the `reserved_only_rejects_non_reserved_peers` test to ensure
litep2p handles peers properly from different states

This PR has been extracted from
#8461 to ease the review
process

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
(cherry picked from commit e82b4db)
paritytech-release-backport-bot Bot pushed a commit that referenced this pull request Jun 4, 2025
…8650)

This PR rejects non-reserved peers in the reserved-only mode of the
litep2p notification peerset.

Previously, litep2p ignored completely the reserved-only state while
accepting inbound connections. However, it handled it properly during
the slot allocation phase.
- the main changes are in the `report_inbound_substream` function, which
now propagated a `Rejected` response to litep2p on the reserved-only
state
- in response, litep2p should never open an inbound substream after
receiving the rejected response
- the state of peers is not advanced while in `Disconnected` or
`Backoff` states
  - the opening state is moved to `Cancelled`
- for consistency purposes (and fuzzing purposes), the
`report_substream_opened` is more robustly handling the `Disconnected`
state
- while at it have replaced a panic with `debug_assert` and an instant
reject

## Testing Done
- started 2 nodes in Kusama and Polkadot with litep2p
- added the `reserved_only_rejects_non_reserved_peers` test to ensure
litep2p handles peers properly from different states

This PR has been extracted from
#8461 to ease the review
process

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
(cherry picked from commit e82b4db)
ordian added a commit that referenced this pull request Jun 4, 2025
* master:
  omni-node: fix `benchmark pallet` to work with `--runtime` (#8594)
  Handle and suppress "New unknown `FromSwarm` libp2p event" warning (#8731)
  Implement detailed logging for XCM failures (#8724)
  [pallet-revive] contract's nonce starts at 1 (#8734)
  sync/fix: Clear gap sync on known imported blocks (#8445)
  [PoP] Add personhood tracking pallets (#8164)
  client/net: Use litep2p as the default network backend (#8461)
  Unflake `returns_status_for_pruned_blocks` (#8709)
  [AHM] Report the weights of epmb pallet to expose kusama and polkadot weights (#8704)
  Remove all XCM dependencies from `pallet-revive` (#8584)
  Docker master image tag fix (#8711)
  Record ed as part of the storage deposit (#8718)
  [pallet-revive] update dry-run logic (#8662)
  feat: add collator peer ID to ParachainInherentData (#8708)
  Nest errors in pallet-xcm (#7730)
  pallet-assets ERC20 precompile (#8554)
  Broker: Introduce min price + adjust renewals to lower market. (#8630)
  [AHM] Staking async fixes for XCM and election planning (#8422)
  Staking (EPMB): Add defensive error handling to voter snapshot creation and solution verification (#8687)
EgorPopelyaev added a commit that referenced this pull request Jun 5, 2025
…nd (#8730)

Backport #8461 into `stable2503` from lexnv.

See the
[documentation](https://github.com/paritytech/polkadot-sdk/blob/master/docs/BACKPORT.md)
on how to use this bot.

<!--
  # To be used by other automation, do not modify:
  original-pr-number: #${pull_number}
-->

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
EgorPopelyaev pushed a commit that referenced this pull request Jun 10, 2025
…nd (#8733)

Backport #8461 into `stable2412` from lexnv.

See the
[documentation](https://github.com/paritytech/polkadot-sdk/blob/master/docs/BACKPORT.md)
on how to use this bot.

<!--
  # To be used by other automation, do not modify:
  original-pr-number: #${pull_number}
-->

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>
pgherveou pushed a commit that referenced this pull request Jun 11, 2025
…8650)

This PR rejects non-reserved peers in the reserved-only mode of the
litep2p notification peerset.

Previously, litep2p ignored completely the reserved-only state while
accepting inbound connections. However, it handled it properly during
the slot allocation phase.
- the main changes are in the `report_inbound_substream` function, which
now propagated a `Rejected` response to litep2p on the reserved-only
state
- in response, litep2p should never open an inbound substream after
receiving the rejected response
- the state of peers is not advanced while in `Disconnected` or
`Backoff` states
  - the opening state is moved to `Cancelled` 
- for consistency purposes (and fuzzing purposes), the
`report_substream_opened` is more robustly handling the `Disconnected`
state
- while at it have replaced a panic with `debug_assert` and an instant
reject
  
## Testing Done
- started 2 nodes in Kusama and Polkadot with litep2p
- added the `reserved_only_rejects_non_reserved_peers` test to ensure
litep2p handles peers properly from different states


This PR has been extracted from
#8461 to ease the review
process

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
pgherveou pushed a commit that referenced this pull request Jun 11, 2025
# Litep2p Becomes the Default Network Backend

This PR finalizes the [litep2p](https://github.com/paritytech/litep2p)
integration and makes it the default network backend for substrate-based
chains.

## Litep2p Improvements

After the stabilization, a forum post will follow with up to date
information and more accurate measurements of the live impact of
litep2p.

### CPU Usage Reduction

**Litep2p consumes roughly 2x less CPU than the libp2p alternative**.
This frees up resources for other usecases (subsystems) and enables
running nodes on more cost-efficient hardware.

This metric has been collected by the `networking::libp2p-node` metric
of a live Kusama validator. This represents the CPU time spent on
polling the networking task. Litep2p CPU consumption is on the left,
using roughtly 1.3x CPUs, while libp2p on the right uses roughly 2.9-3x
CPUs:

![Screenshot 2025-05-26 at 15 23
22](https://github.com/user-attachments/assets/17bf1ed8-b887-423e-b131-f0bbf146919e)


This metric has been collected by the NodeExporter of a live Kusama
validator. Litep2p CPU consumption is on the left, using roughtly 230
CPU units, while libp2p on the right uses roughly 350 CPU units. This
makes litep2p ~1.52 times more effiecient:

![Screenshot 2025-05-26 at 15 24
33](https://github.com/user-attachments/assets/8923cb56-241d-4e1d-9593-33c5def2ff4d)



### DHT Improvements and Authority Discovery

Litep2p is able to discover peers faster via the Kademlia protocol than
libp2p. This behavior manifests in faster discovery times for
validators. For context, libp2p discovers 1K DHT records (authority
records) in approximately 10 minutes, while litep2p discovers them in
just 2.5 minutes (for more info see
#7077 (comment)).

This will improve issues we've seen with libp2p that causes validators
to not receive rewards:
- #8548

### Stable Sync Peers

Litep2p presents a more stable peer count in comparison with the libp2p
backend. This ensures we can sync up faster than libp2p to the tip of
the chain. In an older experiment, litep2p syncs to the tip of the chain
in 526s, compared to 803s for libp2p. The stability of connections shows
improvements for other protocols as well:

![Screenshot 2025-05-26 at 15 01
59](https://github.com/user-attachments/assets/ac3607ba-a551-49e5-9a50-f5150a6b619f)

The previous image shows on the left the litep2p version and on the
right the libp2p version.


### Revert Kusama Enablement
This PR reverts #7866.
Litep2p is now enabled by default, we don't need to selectively enable
it on different chains.

### Litep2p 0.9.5

This release primarily focuses on strengthening the stability of the
websocket transport. We've resolved an issue where higher-level
buffering was causing the Noise protocol to fail when decoding messages.

We've also significantly improved connectivity between litep2p and
Smoldot (the Substrate-based light client). Empty frames are now handled
correctly, preventing handshake timeouts and ensuring smoother
communication.

Finally, we've carried out several dependency updates to keep the
library current with the latest versions of its underlying components.

Fixed:
- substream/fix: Allow empty payloads with 0-length frame
([#395](paritytech/litep2p#395))
- websocket: Fix connection stability on decrypt messages
([#393](paritytech/litep2p#393))

Changed:
- crypto/noise: Show peerIDs that fail to decode
([#392](paritytech/litep2p#392))
- cargo: Bump yamux to 0.13.5 and tokio to 1.45.0
([#396](paritytech/litep2p#396))
- ci: Enforce and apply clippy rules
([#388](paritytech/litep2p#388))
- build(deps): bump ring from 0.16.20 to 0.17.14
([#389](paritytech/litep2p#389))
- Update hickory-resolver 0.24.2 -> 0.25.2
([#386](paritytech/litep2p#386))


### Fix peerset reserve only mode

This has been move in PR:
#8650 for ease of
reviewing.
The PR rejects non-reserved peers in the reserved-only mode of the
litep2p notification peerset.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@Polkadot-Forum
Copy link
Copy Markdown

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/announcing-the-stabilization-of-the-litep2p-network-backend/13712/1

0xbillw added a commit to CESSProject/cess that referenced this pull request Jul 17, 2025
…e2412-6` instead of volatile branch to track `polkadot-sdk`)

The latest release tag for `stable2412` is `polkadot-stable2412-7`. Testing has shown that updating to this version causes frequent errors in RPC calls from light clients (`smoldot`) and existing client SDK code. This may be because this version uses `litep2p` as the default library for `substrate`. watch this: paritytech/polkadot-sdk#8461
0xbillw added a commit to CESSProject/cess that referenced this pull request Aug 14, 2025
…e2412-6` instead of volatile branch to track `polkadot-sdk`)

The latest release tag for `stable2412` is `polkadot-stable2412-7`. Testing has shown that updating to this version causes frequent errors in RPC calls from light clients (`smoldot`) and existing client SDK code. This may be because this version uses `litep2p` as the default library for `substrate`. watch this: paritytech/polkadot-sdk#8461
alstjd0921 pushed a commit to bifrost-platform/polkadot-sdk that referenced this pull request Aug 14, 2025
…work backend (paritytech#8730)

Backport paritytech#8461 into `stable2503` from lexnv.

See the
[documentation](https://github.com/paritytech/polkadot-sdk/blob/master/docs/BACKPORT.md)
on how to use this bot.

<!--
  # To be used by other automation, do not modify:
  original-pr-number: #${pull_number}
-->

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
Franwwenkie added a commit to Franwwenkie/cess that referenced this pull request Sep 7, 2025
…e2412-6` instead of volatile branch to track `polkadot-sdk`)

The latest release tag for `stable2412` is `polkadot-stable2412-7`. Testing has shown that updating to this version causes frequent errors in RPC calls from light clients (`smoldot`) and existing client SDK code. This may be because this version uses `litep2p` as the default library for `substrate`. watch this: paritytech/polkadot-sdk#8461
sakura-kun-88 added a commit to sakura-kun-88/cess that referenced this pull request Sep 20, 2025
…e2412-6` instead of volatile branch to track `polkadot-sdk`)

The latest release tag for `stable2412` is `polkadot-stable2412-7`. Testing has shown that updating to this version causes frequent errors in RPC calls from light clients (`smoldot`) and existing client SDK code. This may be because this version uses `litep2p` as the default library for `substrate`. watch this: paritytech/polkadot-sdk#8461
alvicsam pushed a commit that referenced this pull request Oct 17, 2025
…8650)

This PR rejects non-reserved peers in the reserved-only mode of the
litep2p notification peerset.

Previously, litep2p ignored completely the reserved-only state while
accepting inbound connections. However, it handled it properly during
the slot allocation phase.
- the main changes are in the `report_inbound_substream` function, which
now propagated a `Rejected` response to litep2p on the reserved-only
state
- in response, litep2p should never open an inbound substream after
receiving the rejected response
- the state of peers is not advanced while in `Disconnected` or
`Backoff` states
  - the opening state is moved to `Cancelled` 
- for consistency purposes (and fuzzing purposes), the
`report_substream_opened` is more robustly handling the `Disconnected`
state
- while at it have replaced a panic with `debug_assert` and an instant
reject
  
## Testing Done
- started 2 nodes in Kusama and Polkadot with litep2p
- added the `reserved_only_rejects_non_reserved_peers` test to ensure
litep2p handles peers properly from different states


This PR has been extracted from
#8461 to ease the review
process

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
alvicsam pushed a commit that referenced this pull request Oct 17, 2025
# Litep2p Becomes the Default Network Backend

This PR finalizes the [litep2p](https://github.com/paritytech/litep2p)
integration and makes it the default network backend for substrate-based
chains.

## Litep2p Improvements

After the stabilization, a forum post will follow with up to date
information and more accurate measurements of the live impact of
litep2p.

### CPU Usage Reduction

**Litep2p consumes roughly 2x less CPU than the libp2p alternative**.
This frees up resources for other usecases (subsystems) and enables
running nodes on more cost-efficient hardware.

This metric has been collected by the `networking::libp2p-node` metric
of a live Kusama validator. This represents the CPU time spent on
polling the networking task. Litep2p CPU consumption is on the left,
using roughtly 1.3x CPUs, while libp2p on the right uses roughly 2.9-3x
CPUs:

![Screenshot 2025-05-26 at 15 23
22](https://github.com/user-attachments/assets/17bf1ed8-b887-423e-b131-f0bbf146919e)


This metric has been collected by the NodeExporter of a live Kusama
validator. Litep2p CPU consumption is on the left, using roughtly 230
CPU units, while libp2p on the right uses roughly 350 CPU units. This
makes litep2p ~1.52 times more effiecient:

![Screenshot 2025-05-26 at 15 24
33](https://github.com/user-attachments/assets/8923cb56-241d-4e1d-9593-33c5def2ff4d)



### DHT Improvements and Authority Discovery

Litep2p is able to discover peers faster via the Kademlia protocol than
libp2p. This behavior manifests in faster discovery times for
validators. For context, libp2p discovers 1K DHT records (authority
records) in approximately 10 minutes, while litep2p discovers them in
just 2.5 minutes (for more info see
#7077 (comment)).

This will improve issues we've seen with libp2p that causes validators
to not receive rewards:
- #8548

### Stable Sync Peers

Litep2p presents a more stable peer count in comparison with the libp2p
backend. This ensures we can sync up faster than libp2p to the tip of
the chain. In an older experiment, litep2p syncs to the tip of the chain
in 526s, compared to 803s for libp2p. The stability of connections shows
improvements for other protocols as well:

![Screenshot 2025-05-26 at 15 01
59](https://github.com/user-attachments/assets/ac3607ba-a551-49e5-9a50-f5150a6b619f)

The previous image shows on the left the litep2p version and on the
right the libp2p version.


### Revert Kusama Enablement
This PR reverts #7866.
Litep2p is now enabled by default, we don't need to selectively enable
it on different chains.

### Litep2p 0.9.5

This release primarily focuses on strengthening the stability of the
websocket transport. We've resolved an issue where higher-level
buffering was causing the Noise protocol to fail when decoding messages.

We've also significantly improved connectivity between litep2p and
Smoldot (the Substrate-based light client). Empty frames are now handled
correctly, preventing handshake timeouts and ensuring smoother
communication.

Finally, we've carried out several dependency updates to keep the
library current with the latest versions of its underlying components.

Fixed:
- substream/fix: Allow empty payloads with 0-length frame
([#395](paritytech/litep2p#395))
- websocket: Fix connection stability on decrypt messages
([#393](paritytech/litep2p#393))

Changed:
- crypto/noise: Show peerIDs that fail to decode
([#392](paritytech/litep2p#392))
- cargo: Bump yamux to 0.13.5 and tokio to 1.45.0
([#396](paritytech/litep2p#396))
- ci: Enforce and apply clippy rules
([#388](paritytech/litep2p#388))
- build(deps): bump ring from 0.16.20 to 0.17.14
([#389](paritytech/litep2p#389))
- Update hickory-resolver 0.24.2 -> 0.25.2
([#386](paritytech/litep2p#386))


### Fix peerset reserve only mode

This has been move in PR:
#8650 for ease of
reviewing.
The PR rejects non-reserved peers in the reserved-only mode of the
litep2p notification peerset.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
orbit-caster590p added a commit to orbit-caster590p/cess that referenced this pull request Nov 9, 2025
…e2412-6` instead of volatile branch to track `polkadot-sdk`)

The latest release tag for `stable2412` is `polkadot-stable2412-7`. Testing has shown that updating to this version causes frequent errors in RPC calls from light clients (`smoldot`) and existing client SDK code. This may be because this version uses `litep2p` as the default library for `substrate`. watch this: paritytech/polkadot-sdk#8461
ffarall added a commit to Moonsong-Labs/storage-hub that referenced this pull request Mar 13, 2026
### Notable changes:

* Downgraded rust toolchain version from `1.91` to `1.90`

  ```toml
   # Rust 1.90 is required due to polkadot-sdk stable2503 compatibility.
   # Rust 1.91+ breaks pallet-revive-fixtures compilation with error:
   #   "target-pointer-width: invalid type: string '64', expected u16"
# This is because polkavm-linker generates target specs with
"target-pointer-width": "64" (string),
# but Rust 1.91 (PR #144443) changed this to require an integer:
"target-pointer-width": 64.
# polkadot-sdk stable2503 was tested with Rust 1.84.1; 1.90 is the
latest compatible version.
   channel = "1.90"
  ```

Changes were also caused by this change due to clippy warnings that were
not present in 1.91.

* paritytech/polkadot-sdk#7634:

`DecodeWithMemTracking` trait derivation was added to `RuntimeCall`.
~~This change would have been simply inherited if sp-trie updated their
`CompactProof` type which we use extensively in the codebase.~~

~~To satisfy the bound, proof types at extrinsic boundaries now use
`Vec<Vec<u8>>` instead of `CompactProof` (aliased as
[`CompactProofEncodedNodes`](https://github.com/Moonsong-Labs/storage-hub/pull/671/files#diff-c35139710c5462fa6b2f0c0376c749266f0edfc94da2acf58ddecab8c39de37dR27-R32).
This represents the `encoded_nodes` field of a compact proof which
implements `DecodeWithMemTracking` by the parity-scale-code crate. The
conversion to `CompactProof` is done internally when needed for trie
operations.~~

~~This was one of the two solutions that posed the least amount of code
changes and is easier to change once `CompactProof` implements the new
trait. The other more intrusive way was wrapping the `CompactProof` in a
struct which implements the trait, but this lead to many changes that
were hard to track.~~

> **Note:** This is a **temporary workaround**. Once `CompactProof`
implements `DecodeWithMemTracking` upstream in `sp-trie`, this change
will be reverted to use `CompactProof` directly again.
paritytech/polkadot-sdk#11028

* paritytech/polkadot-sdk#7043

`sp-std` crate is deprecated and all uses of it in the codebase have
been replaced with the `alloc` or `core` equivalent imports.

* paritytech/polkadot-sdk#6140

Wrapped transaction extensions for both parachain and solochain-evm
runtimes with `StorageWeightReclaim`. `StorageWeightReclaim` is meant to
be used as a wrapping of the whole transaction extension pipeline, and
will take into account all proof size accurately.

  ```rust
  /// The TransactionExtension to the basic transaction logic.
pub type TxExtension =
cumulus_pallet_weight_reclaim::StorageWeightReclaim<
    Runtime,
    (
        frame_system::CheckNonZeroSender<Runtime>,
        frame_system::CheckSpecVersion<Runtime>,
        frame_system::CheckTxVersion<Runtime>,
        frame_system::CheckGenesis<Runtime>,
        frame_system::CheckEra<Runtime>,
        frame_system::CheckNonce<Runtime>,
        frame_system::CheckWeight<Runtime>,
        pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
        frame_metadata_hash_extension::CheckMetadataHash<Runtime>,
    ),
  >;
  ```

* **Frontier EVM / `ProofSizeExt` on solochain-evm:**

Frontier EVM (stable2503) introduced `get_proof_size()` host function
calls at multiple points in the EVM runner to measure actual proof size
via the `ProofSizeExt` host extension. If `ProofSizeExt` is present
during block building but absent during block import on syncing nodes,
`get_proof_size()` returns different values on each path, leading to
different gas/weight calculations, different state roots, and digest
mismatch panics.

  To fix this, two changes were made to the solochain-evm node service:

* **Block building**: `ProposerFactory::with_proof_recording` (instead
of `ProposerFactory::new`) to register `ProofSizeExt` during block
authoring.
* **Block import**: `sc_service::new_full_parts_record_import(...,
true)` (instead of `new_full_parts`) so that the Client registers
`ProofSizeExt` during block import on syncing BSP/MSP nodes, matching
the block building path.

- paritytech/polkadot-sdk#6349:

Genesis config preset files have been migrated to use the new
`build_struct_json_patch!` macro instead of the previous pattern of
constructing a full `RuntimeGenesisConfig` struct and serializing it
with `serde_json::to_value()`.

  Before:

  ```rust
  let config = RuntimeGenesisConfig {
      balances: BalancesConfig { ... },
      ..Default::default()
  };
  serde_json::to_value(config).expect("Could not build genesis config.")
  ```

  After:

  ```rust
  use frame_support::build_struct_json_patch;
  build_struct_json_patch!(RuntimeGenesisConfig {
      balances: BalancesConfig { ... },
  })
  ```

* paritytech/polkadot-sdk#8461

Substrate based chains are now configured to use `litep2p` as the
network backend (before was libp2p)

While `litep2p` implements the same wire protocol and is interoperable
with `libp2p` nodes, several behavioral differences required fixes:

* **GRANDPA notification keepalive**: In dev mode with manual sealing,
the GRANDPA voter is not started but its notification protocol is still
registered for P2P negotiation. `litep2p` kills all P2P connections if
any registered notification protocol's service handle is dropped (unlike
`libp2p` which tolerates this). A keepalive task was added that drains
GRANDPA notification events without running consensus, keeping the
protocol handler alive for the lifetime of the node.

* **Chunk uploader retry count**: `litep2p` maps transient connectivity
errors (`ConnectionClosed`, `SubstreamClosed`, `dial-failed`, etc.) to
`RequestFailure::Refused` instead of `RequestFailure::NotConnected`. The
chunk uploader's retry limit for `Refused` errors was increased from 3
to 30 to handle peer recovery scenarios during file uploads.
  
* **Integration tests**: Integration tests were modified to take into
account hew new `Refused` error propagation which changed timing and
logging assertions in the tests.

## ⚠️ Breaking Changes ⚠️


- **Short description**

The transaction extension pipeline (`SignedExtra` / `TxExtension`) must
now be wrapped with
`cumulus_pallet_weight_reclaim::StorageWeightReclaim`. This wrapper
measures PoV size before and after execution and reclaims unused storage
weight, including weight consumed by the extensions themselves. Runtimes
must also implement `cumulus_pallet_weight_reclaim::Config`.

  This affects:
  - The `SignedExtra` / `TxExtension` type alias
- The `from_minimal_extension` implementation (must wrap the inner
extension tuple)
- The `compute_signed_extra_implicit` runtime API implementation (same
wrapping)

- **Who is affected**

* **Downstream runtimes** that integrate StorageHub pallets and define
their own transaction extension pipeline
* **Client code** that implements
`ExtensionOperations::from_minimal_extension`

- **Suggested code changes**

  Type alias:

  Before:

  ```rust
  pub type SignedExtra = (
      frame_system::CheckNonZeroSender<Runtime>,
      frame_system::CheckSpecVersion<Runtime>,
      frame_system::CheckTxVersion<Runtime>,
      frame_system::CheckGenesis<Runtime>,
      frame_system::CheckEra<Runtime>,
      frame_system::CheckNonce<Runtime>,
      frame_system::CheckWeight<Runtime>,
      pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
      frame_metadata_hash_extension::CheckMetadataHash<Runtime>,
  );
  ```

  After:

  ```rust
pub type SignedExtra =
cumulus_pallet_weight_reclaim::StorageWeightReclaim<
      Runtime,
      (
          frame_system::CheckNonZeroSender<Runtime>,
          frame_system::CheckSpecVersion<Runtime>,
          frame_system::CheckTxVersion<Runtime>,
          frame_system::CheckGenesis<Runtime>,
          frame_system::CheckEra<Runtime>,
          frame_system::CheckNonce<Runtime>,
          frame_system::CheckWeight<Runtime>,
          pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
          frame_metadata_hash_extension::CheckMetadataHash<Runtime>,
      ),
  >;
  ```

  `from_minimal_extension`:

  Before:

  ```rust
  fn from_minimal_extension(minimal: MinimalExtension) -> Self {
      (
          frame_system::CheckNonZeroSender::<Runtime>::new(),
          frame_system::CheckSpecVersion::<Runtime>::new(),
          frame_system::CheckTxVersion::<Runtime>::new(),
          frame_system::CheckGenesis::<Runtime>::new(),
          frame_system::CheckEra::<Runtime>::from(minimal.era),
          frame_system::CheckNonce::<Runtime>::from(minimal.nonce),
          frame_system::CheckWeight::<Runtime>::new(),

pallet_transaction_payment::ChargeTransactionPayment::<Runtime>::from(minimal.tip),
          frame_metadata_hash_extension::CheckMetadataHash::new(false),
      )
  }
  ```

  After:

  ```rust
  fn from_minimal_extension(minimal: MinimalExtension) -> Self {
      let inner = (
          frame_system::CheckNonZeroSender::<Runtime>::new(),
          frame_system::CheckSpecVersion::<Runtime>::new(),
          frame_system::CheckTxVersion::<Runtime>::new(),
          frame_system::CheckGenesis::<Runtime>::new(),
          frame_system::CheckEra::<Runtime>::from(minimal.era),
          frame_system::CheckNonce::<Runtime>::from(minimal.nonce),
          frame_system::CheckWeight::<Runtime>::new(),

pallet_transaction_payment::ChargeTransactionPayment::<Runtime>::from(minimal.tip),
          frame_metadata_hash_extension::CheckMetadataHash::new(false),
      );
      cumulus_pallet_weight_reclaim::StorageWeightReclaim::new(inner)
  }
  ```

  New pallet config required:

  ```rust
  impl cumulus_pallet_weight_reclaim::Config for Runtime {
      type WeightInfo = ();
  }
  ```

---

- **Short description**

For EVM runtimes using Frontier, both block building and block import
must now register the `ProofSizeExt` host extension. Frontier EVM
(stable2503) calls `get_proof_size()` at multiple points during EVM
execution. If `ProofSizeExt` is present during block building but absent
during block import on syncing nodes, `get_proof_size()` returns
different values on each path, leading to different gas/weight
calculations, different state roots, and digest mismatch panics.

  Two changes are needed in the node service:
1. `ProposerFactory::new` must be replaced with
`ProposerFactory::with_proof_recording`
2. `sc_service::new_full_parts` must be replaced with
`sc_service::new_full_parts_record_import` with proof recording enabled

- **Who is affected**

* **Downstream node implementations** that run Frontier EVM
(solochain-evm or any EVM-enabled runtime)

Non-EVM runtimes (e.g. parachain-only) are not affected by this specific
change, though using `with_proof_recording` is still recommended for
accurate `StorageWeightReclaim` operation.

- **Suggested code changes**

  Block building — ProposerFactory:

  Before:

  ```rust
  let proposer_factory = sc_basic_authorship::ProposerFactory::new(
      task_manager.spawn_handle(),
      client.clone(),
      transaction_pool.clone(),
      prometheus_registry.as_ref(),
      telemetry.as_ref().map(|t| t.handle()),
  );
  ```

  After:

  ```rust
let proposer_factory =
sc_basic_authorship::ProposerFactory::with_proof_recording(
      task_manager.spawn_handle(),
      client.clone(),
      transaction_pool.clone(),
      prometheus_registry.as_ref(),
      telemetry.as_ref().map(|t| t.handle()),
  );
  ```

  Block import — Client initialization:

  Before:

  ```rust
  let (client, backend, keystore_container, task_manager) =
      sc_service::new_full_parts::<Block, RuntimeApi, _>(
          config,
          telemetry.as_ref().map(|(_, telemetry)| telemetry.handle()),
          executor,
      )?;
  ```

  After:

  ```rust
  let (client, backend, keystore_container, task_manager) =
      sc_service::new_full_parts_record_import::<Block, RuntimeApi, _>(
          config,
          telemetry.as_ref().map(|(_, telemetry)| telemetry.handle()),
          executor,
          true, // enable proof recording on import
      )?;
  ```

---

- **Short description**

StorageHub now targets polkadot-sdk `stable2503` and frontier
`stable2503`. Downstream consumers must align their polkadot-sdk and
frontier dependency versions. This brings several inherited breaking
changes from the Polkadot SDK upgrade.

- **Who is affected**

* **All downstream runtimes and node implementations** that depend on
StorageHub crates

- **Suggested code changes**

* **`sp_std` deprecated**
([polkadot-sdk#7043](paritytech/polkadot-sdk#7043)):
Replace all `sp_std::*` imports with `alloc::*` or `core::*`
equivalents. Add `extern crate alloc;` where needed.

    ```rust
    // Before
    use sp_std::{collections::btree_map::BTreeMap, vec::Vec};

    // After
    extern crate alloc;
    use alloc::{collections::BTreeMap, vec::Vec};
    ```

* **Genesis config presets**
([polkadot-sdk#6349](paritytech/polkadot-sdk#6349)):
Migrate from `serde_json::to_value()` to the `build_struct_json_patch!`
macro.

    ```rust
    // Before
    let config = RuntimeGenesisConfig {
        balances: BalancesConfig { ... },
        ..Default::default()
    };
serde_json::to_value(config).expect("Could not build genesis config.")

    // After
    use frame_support::build_struct_json_patch;
    build_struct_json_patch!(RuntimeGenesisConfig {
        balances: BalancesConfig { ... },
    })
    ```

  * **`pallet_session::Config`**: Add `type DisablingStrategy = ();`

* **`DecodeWithMemTracking`**
([polkadot-sdk#7634](paritytech/polkadot-sdk#7634)):
Custom types used at extrinsic boundaries or in `RuntimeCall` must
derive `DecodeWithMemTracking` from the `parity-scale-codec` crate.

* **`litep2p` default network backend**
([polkadot-sdk#8461](paritytech/polkadot-sdk#8461)):
Substrate chains now use `litep2p` instead of `libp2p` by default. While
`litep2p` is wire-compatible with `libp2p`, behavioral differences exist
(e.g. notification protocol handling, error mapping for
request-response).


<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/moonsong-labs/storage-hub/pull/671"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T0-node This PR/Issue is related to the topic “node”.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants